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ABSTRACT 



This manual from Austin Community College (ACC) (Texas) is a 
guide for faculty and administrators developing assessment criteria for their 
students, classes, or programs. It defines the five stages of ACC's 
assessment process: (1) stating the purpose, wherein the unit or department 

states its function within the larger context of the college's overall 
mission; (2) identifying the outcomes the unit intends to achieve and 
criteria it will use to determine whether those outcomes have been achieved; 
(3) assessing outcomes- -defining the methods by which the unit will assess 
whether the outcomes criteria have been met and analyzing the data gathered 
by that methodology; (4) implementing improvements based on outcomes 
information; and (5) stating the impact the unit's improvements have made on 
the quality of what it does. Completing all five steps ends one assessment 
cycle; each successive year, the steps are completed again. The manual 
recommends that unit members: focus on assessment as a tool for improving the 
quality of students' educational experiences; add minimally to the already 
heavy workload, using existing data as much as possible; ensure that 
attention to institutional effectiveness efforts is not lost by identifying 
the responsibilities of all involved; and be sensitive to the implications of 
assessment. Contains a glossary of assessment terms, Bloom's Classification 
of Cognitive Skills, ACC's mission statement, and an assessment plan 
template. (EMH) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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I. Outcomes Assessment 



Spiraling toward Excellence 



Defining desired outcomes 

Measuring achievement 
of these outcomes 

Using results to improve 



(over... 
and over... 
and over again) 
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Goal 



The ultimate goal of assessment 
is to pose questions and gather information 
to bring about improvement. 



Austin Community College is committed to helping faculty, administrators, and staff find 
ways to ask and answer difficult questions about teaching and learning so that we can con- 
tinue to improve the quality of education. 

Assessment data is generated as we look for answers to questions such as: 

? How do we know if our students are learning what we are teaching? 

? How do we know if we are teaching what our students need to learn? 

? How can we improve the educational success of all students regardless of their 
educational background, race, gender or ethnicity? 

? How well do we prepare students for transfer or the job market? 

? How can we get answers to those questions about student learning that test 
questions don't provide? 

Underlying each of these questions is the assumption of a desire for educational excel- 
lence — excellence in learning, excellence in teaching, and excellence in supporting the 
teaching and learning process. 
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Principles 



Assessment at ACC 

• is faculty or staff driven. Faculty members identify learning outcomes, specify the 
means of assessment, and decide what to do with the results; staff identify 
outcomes for non-instrudional units, specify the means of assessment, and 
decide what to do with the results, 

• is an ongoing process. Instructors, as well as college services personnel, 
assess achievement of outcomes as a regular and annual event, 

• continues after and apart from visits of accrediting agencies because the 
reason for assessment is improvement, 

• refocuses institutional attention on quality, 

• is not and should not be associated with faculty or staff evaluation, 

• analyzes the learning process, not the individual student, faculty member, or 
course. 
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Levels of Assessment 



Assessment at the Classroom Level 

Assessment of student learning begins in the classroom. This is the setting in which, as 
the Latin root of the word assessment suggests, the teacher and student can sit beside each 
other to evaluate learning in a "conversation" that informs improvement. At the classroom 
level, instructors may employ Classroom Assessment Techniques (CATs) to determine the 
effectiveness of their instruction, i.e., the student learning outcomes that are being achieved. 

Assessment at the Course or Program Level 

At the unit level, a similar "conversation" occurs when faculty collaborate to identify, and 
then design assessment plans to evaluate, the critical learning outcomes students should be 
able to demonstrate as a result of completing the program/course. These assessment plans 
are documented in the Unit-Level Effectiveness Database. 

Assessment at the Institutional Level 

And finally, at the institutional level, the "conversation about student learning" occurs in 
the evaluation of the effectiveness of college-wide functions such as transfer and workforce 
programs, retention, student services, and developmental and adult education. These 
comprise the college-wide effectiveness measures. Each is documented yearly in the 
publication of Effectiveness Updates as well as in the Unit-Level Effectiveness Database. 

Common to all of these conversations is the belief that learning is what matters most to 
those of us in community college education. Because of this belief, we assess our 
effectiveness; we seek evidence of the degree to which learning is taking place; and we seek 
information to guide the steps we must take to improve what matters most to us — student 
learning. 
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Process 



The purpose of the assessment process at ACC is to provide concrete information to 
enhance the quality of teaching and learning as well as the quality of services provided in 
support of teaching and learning. The process consists of five "steps," which comprise an 
ongoing opportunity for recursive improvement that supports our quest for excellence. 



State 

Purpose 



1. The unit states its function within the larger context of the College's overall 
mission. 




Identify 

Outcomes 



2. The unit identifies the outcomes it intends to achieve and criteria it will 
use to determine whether those outcomes have been achieved. 



Assess 3. The unit defines the methods by which it will assess whether the 
Outcomes outcomes criteria have been met and analyzes the data gathered by 
that methodology. 



Implement 
Improvements 



4. The unit creates and implements improvement plans based 
on the information it gathered from assessing its outcomes. 




State 

Impact 



5. The unit states the impact its improvements have 
made on the quality of what it does. 



These steps are dynamic and interactive. The unit's purpose is made concrete and 
specific in the outcomes it identifies. The outcomes, in turn, are directly addressed by the 
methods and criteria designed to demonstrate achievement of those outcomes, and 
improvements are implemented that impact the quality of the unit's function. Completing all 
five steps ends one assessment cycle. Each successive year, the steps are completed again. 
Using information and improvement plans from the previous year as the basis for assess- 
ment planning the following year creates a continuous quality improvement spiral that 
supports each unit's efforts to improve the educational experience of students at ACC. 
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Definitions 



The following terms are defined as they are used in the assessment process at ACC. 

Purpose: The unit's purpose is a broad statement identifying the major function of the unit. 

EX 1: The Legal Assistant Program consists of a sixty-six hour training program for 
those desiring to function as legal assistants (paralegals) in a law office or legal 
agency or section. Upon successful completion of the program, the student receives 
an Associate of Applied Science Degree as a Legal Assistant. Although students can 
transfer the degree to some four-year universities in order to obtain a baccalaureate, 
the program is designed to prepare students for immediate entry into the workforce. 
The purpose of the program is to prepare the student to function successfully as a 
legal assistant or paralegal in a law environment. 

EX 2: The purpose of the Office of Institutional Effectiveness is to coordinate the proc- 
esses that lead to effective decision-making, college-wide. This involves serving Col- 
lege units by providing support in the design, data collection and analysis of activities 
assessing the effectiveness of the unit and the College; by coordinating and providing 
data for both operational and strategic planning activities; and by fulfilling College ex- 
ternal reporting requirements for accreditation and funding. 

Goal: A goal is a broad, future-oriented statement identifying what the unit is striving to- 
wards, hoping to become or proposes to do to fulfill its unit purpose. Goals are used primar- 
ily in policy making and planning. 

EX 1: The Legal Assistant Program will provide students with the necessary 
educational experiences to allow them to function successfully as a legal assistant or 
paralegal in a law environment. 

EX 2: The OIE will provide support to College units in the design, data collection and 
analysis of activities assessing the effectiveness of the unit and the College; will coor- 
dinate and provide data for both operational and strategic planning activities; and 
will fulfill College external reporting requirements for accreditation and funding. 

Outcome: An outcome is the brief, clear statement identifying in measurable terms the 
intended result of processes and services of the unit. Outcomes focus on the specific per- 
formance^) stakeholders are expected to demonstrate when the unit achieves its goal. 

EX 1: Students graduating from the ACC Legal Assistant Program will be prepared to 
function as entry-level legal assistants or paralegals. 

EX 2: Faculty and staff will have the information they need. 
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Criteria: A criterion is a statement of the specific standards that identify successful 
achievement of the intended outcome, including the data that will be used, who will 
demonstrate achievement of the outcome, when, under what conditions, and to what 
degree. 

EX 1: la. Students taking the Internship Course during their last or next-to-last 
semester of the program will be evaluated by a potential employer as adequately 
prepared to function as an entry-level legal assistant. 

lb. Less than 5% of the graduates of the Program will return to ACC to seek re- 
training in an area in which they completed a course with a grade of "C" or better. 

EX 2: 95% of faculty and staff responding to the annual OIE evaluation survey will re- 
port they have used information from the OIE in making decisions affecting their pro- 
gram. 

Methodology: The methodology is a description of the actions that must be taken to gather, 
analyze, and report the findings of the data that will be used to determine whether the 
outcome has been achieved and who is responsible for each. 

EX 1: la. Employers of interns working in a law office or other legal environment are 
asked at the end of the internship to evaluate the intern's performance in nine 
different areas and to determine whether the intern is adequately "prepared to 
function as an entry-level legal assistant." The evaluation scale is 1 to 5, with 1 being 
"inadequate," 2 being "needs improvement," 3 being "adequate," 4 being "very 
good," and 5 being "superior." 

lb. Records of any graduate seeking retraining are kept. A percentage 
comparing the number of graduates to the number of students seeking retraining will 
be calculated each semester. 

EX 2: The Coordinator of Institutional Assessment will calculate the percentage of 
faculty and staff responding "yes" to the following survey item, "In the last year, have 
you used information from the OIE to make decisions regarding your program?' 



Direct Indicator: Direct indicators of learning are immediately observable. Direct indicators 
of learning include pre-and post-testing; capstone courses; oral examinations; internships; 
portfolio assessments; evaluation of capstone projects; standardized national exams; locally 
developed tests; performance on licensure, certification, or professional exams; and juried 
reviews and performances. 

Indirect Indicator: Indirect indicators of learning are subsequently observable. Indirect indi- 
cators might include information gathered from alumni, employers, and students; graduation 
rates; transfer studies, graduate follow-up studies; success of students in subsequent institu- 
tional settings; and job placement data. 
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Qualitative Assessment Tools: Qualitative assessment tools attempt to grasp the whole of a 
student's achievement with information which usually cannot be quantified or counted. How- 
ever, when scoring rubrics (or criteria) are applied to qualitative assessments, they can provide 
quantitative data for program assessment. 

Quantitative Assessment Tools: Quantitative assessment tools produce numerical data which 
can easily be aggregated to indicate program performance. Examples of assessment tools that 
provide quantitative data include standardized tests, locally-developed tests, licensure exams, 
surveys, etc. 



Thoughts to Keep in Mind when Creating Assessment Plans 



1. Focus on assessment as a tool for improving the quality of students’ educational 

experiences. 

2. Add minimally to the already heavy workload. 

• Stress those things that are essential and that will facilitate decisions that the program 
is prepared to make. 

• Collect only information that will be used. 

• Use existing data as much as possible. 

• Incorporate assessment procedures into routine operation of the functional unit. 

• Use a cycle of assessment so that every outcome is assessed within a 3 - 5 year 
period. 

• Coordinate surveying efforts. 

3. Ensure that attention to institutional effectiveness efforts is not lost by identifying the 
responsibilities of all involved. 

4. Be sensitive to the implications of assessment. Data reported insensitively may do more 
harm than good. Internally, results of assessment should be seen as formative, not punitive. 
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II. Documentation of 
Outcomes Assessment 
at ACC: 



"How to....” for Instructional Units 
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Overview 



This "How to" section is intended to provide assistance to instructional units as they begin the 
process of documenting their program's assessment plans. Documentation is entered into 
the Unit-Level Effectiveness Database at http : //www2.austin.cc.tx.us./oiepub/unitlevel.htm. The user 
name for entering documentation into the database is ACC Units (two words) and the pass- 
word is unitgoals (one word.) 

The format for this documentation includes the following seven steps: 

1. Stating the purpose of the unit 

2. Identifying the outcomes of instruction that indicate student learning 

3. Defining the criteria that will determine whether the outcomes have been achieved 

4. Identifying the methodology to be used to gather data identified in the criteria 

5. Summarizing results 

6. Reporting improvement actions to be (or that have been) taken based on the results. 

7. And finally, reporting the impact of implemented improvements 



In the pages that follow, each of these steps will be discussed, and the unit's answers to key 
questions (marked by this symbol ( «-* ) will identify the essential elements required for 
documentation of each step. 

For additional assistance contact Roslyn Wallace (rwall@austin.cc.tx.us or 223-7585). 
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1. Purpose 




1 ? 



Why does the program exist? 

What does the program provide students? 




The first step in any assessment plan is to identify the purpose or "mission" of the "unit" 
in terms of the function it fulfils within the College's mission. The units purpose is the founda- 
tion of the assessment plan and thus, provides the context for each of the steps in the as- 
sessment process. 



Begin with the end in mind. 

T ake a moment and picture a clear bowl more than half filled with pebbles. 

The bowl represents a typical week and the pebbles represent the little tasks that we can see 
we already have to do. Next to the bowl are some fist-sized river stones. These stones have la- 
bels on them with names such as Class, Study Time, Major Publication, Family, Sleep, and 
Grant Application. The task is to put the big rocks into the bowl so that the lid goes on. When 
faced with this task, most people attempt to wedge the big rocks into the little ones. This ap- 
proach represents how most people try to achieve their week's work: They wedge important 
tasks into already scheduled smaller ones, rearranging the smaller ones around the bigger ones. In 
contrast, if presented with a second, empty bowl next to the first, a different paradigm arises. 
Now, people put the big rocks into the second bowl and then pour the pebbles in on top, where 
they fill in the empty spaces between the rocks. The lid goes on easily. 

What do we learn from such an exercise? The metaphor permits many interpretations. I like 
this one: "BIG ROCKS GO IN FIRST." If the big rocks are put in at the beginning, they 
all fit.. .and even if the pebbles don’t all get in, who cares? They're not "Big Rocks." When Big 
Rocks Go In First, every week is a success. 

When people or agencies write down what their Really Big Rocks are over time, they create a 
document that is sometimes called a mission [purpose] statement. A mission statement illuminates 
the question, "Why are we here?" A statement of goals translates the mission statement into 
promises and actions. From an assessment perspective, it's a good idea to know what the Big 
Rocks are for our students— and to inform them about them, too! — because these are what we 
care about and what we assess first. 

"If you don't know where you're headed, you'll probably end up someplace else." 

Douglas J Eder, 
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Picture the program for which you have responsibility. What Big Rocks related to student 
learning do you want to "go in first”? That is, what major goals do you wish your students to 
achieve under your guidance? 

Faculty may spend a great deal of time coming to agreement on the mission/purpose 
statement, as each may have his/her own idea of the characteristics of a purpose state- 
ment — long, short, values oriented, a slogan. Keep the Big Rocks in mind, and come to con- 
ceptual agreement among members of the unit on who the unit serves, in what ways, and 
with what results. Keep in mind that usually the unit purpose is stated in the Catalog and 
follows the phrase, "prepares the student to....” 

EX: The Legal Assistant Program consists of a sixty-six hour training program for 
those desiring to function as legal assistants (paralegals) in a law office or legal 
agency or section. Upon successful completion of the Program, the student receives 
an Associate of Applied Science Degree as a Legal Assistant. Although students can 
transfer the degree to some four-year universities in order to obtain a baccalaureate, 
the Program is designed to prepare students for immediate entry into the workforce. 
The Purpose of the Program is to prepare the student to function successfully as a 
legal assistant or paralegal in a law environment. 

9- Who does the unit serve? 

Obviously, the unit exists to serve students, but the mission/purpose statement should 
identify specific major stakeholders — those who will be effected by the activities of the unit — 
and "consumers" of the unit's unique "product." Employers, as well as four-year institutions, 
and even the community, may be stakeholders. The point is to focus the unit's attention on 
the people it intends to serve. 

EX: "those desiring to function as legal assistants (paralegals) in a law office or legal 
agency or section" 

9- What unique product or service does the unit provide its stakeholders? 

The answer to this question identifies what the unit does and is the basis for its outcomes 
statements. 

EX: "consists of a sixty-six hour training program . . ..Upon successful completion of 
the Program, the student receives an Associate of Applied Science Degree as a Legal 
Assistant" 

What is the result of the program? 

The answer to this question identifies what should be the result to the stakeholder when the 
unit accomplishes its mission/purpose. It is the "bottom line" for the unit and for its 
stakeholders. 

EX: "The program is designed to prepare students for immediate entry into the 
workforce" 

Once the unit's purpose statement has been identified, enter it into the Unit-Level Effec- 
tiveness Database http://www2.austin.cc.ix.us./oiepub/unitlevel.htm. The User name for entering 
documentation into the database is ACC Units and the password is unitgoals. 
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Quality Check: Purpose Statement 



(jg^ The purpose statement identifies the unit's major "stakeholders." 
<S§f The purpose statement identifies the unique "product" of the unit. 
<§f The purpose statement identifies the major "task" of the unit. 



2. Outcomes 



• Who will demonstrate the results of the units achievement of its purpose? 

&—T • What specific behavior or action will demonstrate these results? 

• What will students know or be able to do? 

• What skills will students have acquired? 

• What attitudes, values, or interests will students demonstrate? 

• When will the results be demonstrated? 



I 



Learning outcomes are the vehicle for determining whether the unit's purpose is be- 
ing achieved. Outcomes information provides concrete, meaningful, and useful evidence of 
achievement of the units purpose, i.e., whether the unit is doing what it proposes to be do- 
ing. Well designed outcomes also provide information to direct improvement, if such is 
needed. Thus, a unit's instructional outcomes serve as the foundation for assessment 
planning. Outcomes statements 

• provide direction to the program, 

• communicate what the faculty in the program deem important, 

• identify the intended results of the educational program, 

• describe what students should be able to know, think, or do, 

• describe the attitudes, values and skills to be acquired by completing an 
educational program, 

• are indicators of the effectiveness of a program. 

Outcomes statements are needed before appropriate assessment tools and 
procedures can be selected or designed. Having outcomes statements that are both clear 
and measurable greatly facilitates the identification of appropriate assessment techniques. 

Essential Components of Learning Outcomes 

The essential components of outcomes statements are described below. After a 
component is defined, the phrase that illustrates it will be extracted from the following 
outcomes statement: 

After analyzing and interpreting information from public opinion polls, the 

graduating journalism major will be able to communicate the results to at least 

three different groups in written, oral, and graphic forms. 

Who will demonstrate results? 

Specify subgroups. 

• students majoring in the program 

• students taking specific courses to meet requirements for other programs 

• graduates of the program 

• employers of program graduates 

EX: graduating journalism major [student] 

15 
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8— ' What specific behavior or action will demonstrate results? 

Specify actions or behaviors that follow instruction and could serve as evidence that the 
objective has been achieved. 

• Use action verbs 

• Identify the "focus" of learning-content, concept, skill, attitude 
Make sure the action or behavior is specific enough to be measurable. 

EX: communicate results in written, oral, and graphic forms to at least three 

different groups 

When will results be demonstrated? 

Give information about situations in which the student will be required to demonstrate 
the behavior 

• at the beginning and end of a course or the program 

• following a specific course 

• following graduation 

• following a specific task 

EX: After analyzing and interpreting information from public opinion polls 

Outcomes vs. Goals 

There may be a tendency to create outcomes that describe instructional activities /’goals) 
rather than student learning (outcomes). Although goals and outcomes seem similar, some 
differences do exist. 



Goals 

broad, future-oriented statements 

reflect what the program is striving towards or 
hoping to become 

refer to instructional processes/activities 

used primarily in policy making and general 
program planning 



Outcomes 

brief, clear statements of results 

describe the desired learning/behaviors of students 
completing the program or course 

refer to results of instructional processes/activities 

used to assess effectiveness of curriculum and pro- 
vide information for improvements 



At a minimum, outcome statements should describe student behaviors and products 
that faculty would accept as evidence that learning was achieved, thus providing documenta- 
tion of the effectiveness of the curriculum. Goal statements are helpful for directing instructional 
activities, but are sometimes too general, broad, or vague for developing specific tools to assess 
student learning. Goals provide the context for outcomes. 
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Development of Learning Outcomes 

Often outcomes are readily available in the unit. However, they may need to be put into 
written form, revised, or updated. When outcomes statements do not already exist, there are a 
number of processes that can be used to identify and develop them. 

• Examine mission and goal statements, as well as other materials that describe the 
program. A linkage should exist belween the mission of the college and that of the unit. 
Likewise, there should be a connection between a unit's purpose and the outcomes of 
the program. 

• Use faculty discussion sessions and/or surveys. This can be done in a couple of ways. 
The obvious one is to ask faculty what students should be able to do before they 
complete the program. Another way would be to ask faculty to visualize the ideal 
student that has just completed the program. Then ask them to describe what the 
student knows (cognitive), can do (cognitive, psychomotor), and cares about (affective). 
Finally, faculty could discuss what achievements are expected of students that have 
completed the program. 

• Collect and review outcomes from similar programs and external sources. This is one 
way to determine what is distinctive about a particular program. Discipline-specific 
professional societies and accrediting bodies can be good sources of outcomes . Before 
adopting an outcome from an external source, the faculty should determine whether the 
outcome is appropriate for their program based on the student characteristics, 
resources, curriculum, etc., associated with the program. 

• Use focus group discussions and surveys. Ask students (entering and current) what their 
expectations and goals are. Alumni could be asked to describe what they felt they 
gained from the program. Both alumni and employers could be asked what outcomes 
they feel are important for the future employment of the program's graduates. By 
consulting these groups, faculty broaden their information base for identifying outcomes, 
and the groups may be more motivated to participate in other evaluation/assessment 
activities. 

• Describe the ideal student in your program at various phases throughout your program. 
Be concrete and focus on those strengths, skills, and values that you feel are the result 
of, or at least supported and nurtured by, the program experience. Then ask: What does 
this student know? What can this student do? What does this student care about? 

• List and briefly describe the program experiences that contribute most to the 
development of the ideal student. 

• List the achievements you implicitly expect of graduates in each major field. 

• Describe your alumni in terms of such achievements as career accomplishments, 
lifestyles, citizenship activities, and aesthetic and intellectual involvement. 

• Collect and review instructional materials. Try sorting materials into 3 broad categories: 
recognition/recall, comprehension/simple application, critical thinking/ problem-solving. 

17 
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Use any of the following: syllabi and course outlines, course assignments and tests, 
textbooks (especially the tables of contents, introductions, and summaries). 

• Collect and review documents that describe your department and its programs: 
brochures and catalog descriptions, accreditation reports, curriculum committee reports, 
mission statements. 

These activities should result in a plethora of possible outcomes to assess the effectiveness 
of the unit. It is suggested, however, that only three to five objectives be assessed each year. 
Reasons for this limitation are primarily based on the availability of resources — time, staff, ability 
of the unit to address the results of assessing the outcomes, etc. To refine or reduce a set of 
outcome statements, use the 25 percent problem. Imagine that you want to reduce program or 
course material by 25 percent. What outcomes would you keep and which would you discard? 

Select outcomes that 

• faculty agree are the most important, 

• have the potential to make a difference in how the unit functions, 

• will yield information that will help a unit learn about and improve itself, 

• will insure the assessment plan will actually be implemented. 

When the unit's outcomes have been identified, enter them into the Unit-Level Effectiveness 
Database at http://wvwv2.austin.cc.tx.us./oiepub/unitlevel.htm. The User name for entering documen- 
tation into the database is ACC Units (two words) and the password is unitgoals (one word.) 
Deadline for entering documentation into the ULE Database is June 15 of each year. 




18 



21 



Quality Check: Outcomes Statements 

<§f The outcome describes a learning result rather than a teaching process. 

<§f The outcome describes what the student will be able to demonstrate. 

<§f The outcome is measurable. 

(Jgf The outcome is specific. 

0 The outcome addresses no more than a single result (uses no conjunctions!). 

0 The outcome uses action verbs that specify definite, observable behaviors. 

df The outcome is dear: faculty, students, administrators, and people outside the unit are 
able to understand it. 

0 The outcome is validated by departmental colleagues. 

0 The outcome is clearly linked to unit goals. 

The outcome is reasonable, given the ability of the students. 
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3. Criteria 
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Who will demonstrate results? 

When will results be demonstrated? 

What specific behavior or action will demonstrate results? 
How will results be demonstrated? 

What level of achievement is acceptable to identify success? 



Outcomes identify the results students will demonstrate. Criteria delineate the specific stan- 
dards and parameters the unit has imposed on the outcome and serve to direct the collection of 
data. It is crucial to identify the specific data needed to determine whether the outcome has 
been achieved — 1) who, 2) when, 3) what, 4) how, and 5) how much. These five elements are 
essential in writing a useful criterion statement. Until this step is completed, data cannot be 
gathered to provide information regarding the effectiveness of the unit and the actions it must 
undertake to improve. 

o-’ Who will demonstrate results? 

The criteria must identify the persons whose performance will be measured to determine 
whether the outcome has been achieved. Students majoring in the program, students taking 
specific courses to meet requirements for other programs, graduates of the program, employ- 
ers of graduates, etc., each are possible groups whose performance would demonstrate 
achievement of an outcome. 

®“> When will results be demonstrated? 

When results will be demonstrated depends on the outcome that is being assessed. If end- 
of-program outcomes are being assessed, then the results could be demonstrated at the end of 
the program or following graduation. If the outcome is about performance of graduates in 
transfer institutions or the workforce, the results must be demonstrated after students have 
transferred or been employed. If the outcome examines change over time, the results may 
need to be demonstrated both at the beginning and end of the course/program. If a specific 
skill or task demonstrates achievement of the outcome, the results could be demonstrated fol- 
lowing the time that skill or task is expected to be mastered. The criteria must specify a time 
when results will be demonstrated. 

What specific behavior or action will demonstrate results? 

The criteria must specify the performance that determines achievement of the outcome. Will 
students take a test, complete a survey, perform a task? Use action verbs to describe the results 
that will demonstrate achievement of the outcome. Specify the focus of learning — content, con- 
cept, skill, attitude. 

What level of performance is acceptable to identify successful achievement of the out- 
come? 

The acceptable level of performance that identifies achievement of the outcome is usually a 
numeric expression, such as a percentage. Additionally, the level of success that is acceptable 
must be realistic, i.e., achievable by the persons being assessed and still challenging. There is 
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no minimum required level of achievement to identify success-that is the judgment of the fac- 
ulty members who identify the outcome and criteria. However, performance levels should reflect 
what OUR students can do. "100% surpassing national norms" is too high and unrealistic, ”25% 
of a national norm" is so low as to be suspect. 

There are several types of standards for determining successful achievement of an outcome. 
(NOTE: In the examples below, the numbers identify each of the elements of the criterion statement: 1. who, 2. 
when, 3. what, 4. how, and 5. how much.) 

One type of standard might be a percentage of students projected to achieve some speci- 
fied level of performance. For example, it is quite acceptable for an outcomes statement to say, 
"graduates of XXXX will be successfully employed in the field", but the criteria MUST specify 
what will identify success in achieving this outcome. For example: " 80% (51 of the graduates of 
XXXX program (1) will report employment in the field (31 on the survey (41 distributed one year af- 
ter graduation (21 ." 

Another standard might be a comparison of the level of one group's performance with an- 
other, either within the college or externally. For example, "The average score (3) of the gradu- 
ates of XXXX program 111 on the national XXXX exams (4) administered each spring (21 will be 
±2% of the 50th percentile compared to the national results.(5l " 

Still another standard of successful achievement of an outcome might be performance in 
subsequent endeavors (courses, programs, employment). Examples include, " 90% (5) of those 
students responding ID to a follow-up survey (4) one year after transfer (2) will respond that all of 
their ACC courses were accepted as prerequisites, (3) " and " 75% (51 of students completing de- 
velopmental math with a grade of 'C or better (1) will, within two years (2). complete college al- 
gebra. trigonometry, or topics in math with a grade of 'C' or better (3 & 41 ." 

In some cases, a program may be unable to establish specific criteria before implementing 
an assessment plan. In such cases, the criteria may state something like this, "The first two cy- 
cles of this assessment will be used to establish base-line criteria for success." 

It is important that there be more than one criterion identified for each outcome. No single 
criterion provides a complete view of a single outcome . For example, both a standardized test 
(if available for that program) and a survey of student attitudes which asked if students 'felt well- 
prepared' might assess a 'knowledge-of-the-field' outcome. 

It is equally important that the data to be gathered is, in fact, available. Nothing frustrates 
completion of an assessment plan more than delineating specific standards only to discover 
there is no way to collect the data required for demonstrating those standards. Defining criteria 
that will require the creation of elaborate data collection methods should be undertaken only af- 
ter a serious consideration of costs, benefits, and feasibility. 

When the criteria have been delineated, enter them into the Unit-Level Effectiveness Data- 
base at http://www2.austin.cc.tx.us./oiepub/unitlevel.htm. The User name for entering documentation 
into the database is ACC Units (two words) and the password is unitgoals (one word). 
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Quality Check: Criterion Statements 



(§f The criterion statement identifies who will demonstrate the outcome. 

<§f The criterion statement identifies when the outcome will be demonstrated. 

<§f ’ The criterion statement identifies what behavior or action will demonstrate the outcome. 

(§f The criterion statement identifies how the outcome will be demonstrated. 

(gf The criterion statement identifies the level of performance that is necessary to determine 
successful achievement of the outcome. 
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4. Methodology 






What "tool" will be used to collect the data? 
When will the data be collected? 

How will the data be analyzed? 

How and to whom will the data be reported? 
Who will be responsible? 



The Criteria have identified the data necessary for determining whether the outcomes 
have been achieved. The Methodology moves assessment from planning to implementa- 
tion. It is the "recipe" for collecting the data. It describes the actions that must be taken (and 
by whom) to determine whether the criteria have been met. Together, the criteria and meth- 
odology provide the means with which to adequately, accurately, and specifically document 
achievement of the outcomes. 

Methodology statements identify 

• which tools will be used to gather the data 

• which data will be gathered, by whom, how and when 

• what will be done with the data once they have been gathered. 

In describing the methodology, keep in mind assessment is concerned with the aggre- 
gate performance of a group of students rather than individual students. Also, consider what 
data are readily available in the department or college. 

And, especially remember, "You can't fix in analysis what you bungled in design, " Richard 
Light, By Design. 

Which tools or instalments will be used to collect the data identified in the criteria? 

Adopt, adapt or construct. It may be possible, after serious review, for a unit to adopt a 
commercially available instrument to use for assessing its outcomes. Another alternative 
would be to adapt instruments from another area or use. In many cases, the best solution 
to insure fit between an instrument and a unit's needs involves the construction of an instru- 
ment, whether it is a local test or a questionnaire. 

The only way to determine how useful a tool will be for measuring an outcome of a par- 
ticular unit is to pilot test it. In a sense, all assessment is a pilot test of the methodology be- 
ing used. Some evaluation of the methodology's utility occurs, informally if not formally, each 
time the assessment cycle is completed, and the unit learns whether the methodology actu- 
ally provided the data needed to determine successful achievement of the outcome. 

A variety of data collection methods may be used. The methodology should include at 
least one direct measure of learning for each outcome. A direct indicator of learning is im- 
mediately observable. 
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Direct indicators of learning include 
d pre-and post-testing, 

d capstone courses, 
d oral examinations, 
d internships, 
d portfolio assessments, 
d evaluation of capstone projects, 
d standardized national exams, 
d locally developed tests, 

d performance on licensure, certification, or professional exams, 
d juried reviews and performances. 

An indirect measure of learning may be included in the methodology as a second assess- 
ment technique. Indirect indicators of learning are subsequently observable. Indirect indicators 
might include 

i information gathered from alumni, employers, and students 
i graduation rates 
i transfer studies 
i graduate follow-up studies 
i success of students in subsequent institutional settings 
i job placement data 

There are many, many, many tools available for collecting assessment data. Following is a 
list of commonly used assessment tools for programs and courses. The list is not exhaustive and 
is not a shopping list; that is, departments are urged to create and use assessment tools which 
are NOT on the list whenever the new creation is better suited to measuring the outcome they 
have identified. 

Qualitative Assessment Tools 

These assessment tools try to grasp the whole of a student's achievement with information 
which usually cannot be quantified and/or counted. Typically these tools are scored using 
scoring rubrics. 

• Portfolio - Each student would create a portfolio with the same ingredients and the same 
elements in each would be reviewed. 

• Public Performance - In art or music each student might have a culminating product or 
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performance and, with some creative thought, students might be evaluated in other fields 
by different forms of performance, e.g., participation in student elections, volunteer work, 
dinicals, practice-teaching, etc. 

• Juried Competition - Here there would need to be outsiders, e.g. a panel of colleagues 
from another institution or employers. 

• Interviews - These can be especially useful for college services, where student perception 
of advice/help received can demonstrate performance. 

Quantitative Assessment Tools 

These tools have the advantage of producing numerical data which can be easily aggre- 
gated to indicate program performance. 

• Standardized Tests - These have the advantage of instant credibility but are not available 
for all programs and disciplines and, more importantly, may not cover what is done in the 
program. They are also expensive. 

• Locally-developed Tests - These allow a department to tailor the test to the actual content 
of a program or course and let the faculty later pinpoint just what students aren't achiev- 
ing, but they take a lot of faculty time and may lack credibility. However, they can be useful 
in conjunction with other means of assessment. 

• Licensure Exams - CPA, nursing board, and similar exams are excellent measures of a 
program's effectiveness. 

• Surveys - Sent to former students, these can provide measures of employment, indicators 
of student satisfaction with training received, self-evaluation of competency, etc. Sent to 
employers, they can produce evaluations of student skills, attitudes, and knowledge and 
employer willingness to hire our students in the future. 

• Tracking Data from Transfer Institutions - Texas A&M has provided "Foundation Course 
Tracking" reports to ACC in the past and hopefully will continue to do so in future. These 
data provide information on the performance of students who take a foundation course 
at ACC, then transfer to TAMU and take the target course associated with the foundation 
course. 

• Observational - This assessment tool involves a particular kind of knowledge-based per- 
formance, where the assessment is a matter of observing (or counting) how many stu- 
dents successfully performed some product of learning (assuming it is required of all stu- 
dents in that program or course.) Examples would include creating an executable com- 
puter program, publishing a journal article, etc. 



a- When will the data be collected? 

Timing the collection of data is critical. Consider the rhythm of the academic year when de- 
ciding when to collect data. For example, surveys conducted around the time of holidays tend 
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to have poor response rates. Likewise, the sometime during the last few weeks of a semester 
would be the best time to collect end-of-course performance data. 

^ How will the data be analyzed? 

Consider whether the data will have inherent meaning or be meaningful only when com- 
pared against other data, benchmarks, baselines, etc. Also consider whether the analysis of 
data will be done within the unit or will need to be done by an external agent. These decisions 
will impact the time it takes to translate the data the unit has gathered into useable information. 

^ How and to whom will data be reported? 

Obviously, if the assessment results are to be used to make improvements in instruction, 
they need to be reported to those who are responsible for creating and implementing those im- 
provements. These people need to be identified. Be sensitive to the implications of assessment. 
Data reported insensitively may do more harm than good. Internally, results of assessment 
should be seen as formative, not punitive. 

^ Who will be responsible for collecting the data specified in the criteria? 

The unit should identify someone to "conduct" the assessment. If a "home-made" instru- 
ment is to be used to collect data, who will create it? Who will administer it? If data available 
elsewhere in the College is to be used, who will request/obtain it? Who will analyze the data 
and report the findings? Who will act on the information to make improvements? The unit must 
assign each of these actions to someone (and notify them of the assignment). Without this step, 
the assessment of outcomes becomes an empty exercise that has used faculty time but pro- 
vided nothing useable in return. 

When the unifs methodology has been identified, enter it into the Unit-Level Effectiveness Data- 
base at hftp://www2.austin.cc.tx.us./oiepub/unitlevel.htm. The User name for entering documentation 
into the database is ACC Units (Iwo words) and the password is unitgoals (one word.) Docu- 
mentation ensures the same methodology is used each time the outcome is assessed, thus al- 
lowing for trend analysis in future years. The deadline for entering documentation into the ULE 
Database is June 15 of each year. 
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Quality Check: Methodology 



@ The methodology identifies who will be responsible for conducting the assessment. 

The methodology identifies when the data will be collected . 
d&f The methodology identifies what tool will be used to collect data. 

<§f The methodology identifies how the data will be analyzed. 

<§f The methodology identifies how and to whom the results of the analysis will be reported. 
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5. Results 
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What did the unit find out? 

Were the criteria met? 

What do these findings mean to the unit? 
What problems need to be addressed? 



H • What successes were identified? 

MragsagggMSfflgsagiffiifiiSSgBS^ 



s- What did the unit find out? 

Once the methodology has been implemented and the unit has received the results of 
the assessment, those results should be documented into the Unit-Level Effectiveness Data- 
base. The results need not be full-blown reports; they should be a summary of the findings. 
For example, "Seventy percent (70%) responded they were satisfied" or 'The average was at 
the 48th percentile" are sufficient. Because the criteria were stated in specific terms, the re- 
sults need only be reported in the terms specified in the criteria. 

s- What do these findings mean to the unit? 

It is important that the meaning the unit makes of the finding also be included in the 
statement of results. Again, there is no need to write a dissertation. A brief statement or two 
indicating the significance of the findings to the unit is all that is necessary. For example, 
'These results indicate students are not retaining information from the foundation course." 

The results and their meaning should be documented into the Unit-Level Effectiveness 
Database at http://www2.austin.cc.tx.us./oiepub/unitlevel.htm as soon as they have been made 
available. The deadline for entering documentation of results into the ULE Database is June 
15 of each year. The User name for entering documentation into the database is ACC Units 
(two words) and the password is unitgoals (one word.) 
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6. Improvements 



l$ BBEBBSSBBBZZBEBBZE3EgZ2WSBE^SBBBBBB2SeB2SSZiBWBBBBEBZEB2BBBBeBB22mBmmzmB2^MB^mMZZL 

• What will the unit do regarding the assessed outcome? 

8 — *• • What changes need to be made? 

• What other data are needed to make improvements? 

• What resources are needed to make improvements? 
When will changes be made, and who is responsible? 



This step is where the assessment process comes full circle; where the assessment plan be- 
gins to "pay off'." If the criteria were dear, each result should indicate that the level of learning 
intended to take place either is or is not being achieved. 

8-* What will the unit do to address the results of their assessment of the outcome? 

If the findings indicate the criteria have been met or surpassed, the unit may decide that no 
change is needed and report "Outcome will be assessed once more next year to validate re- 
sults." Then when the outcome is assessed again and the results are repeated, the unit should 
move on to assess another outcome. 

On the other hand, if the findings indicate the criteria have not been met, the results of the 
assessment must be examined further. 

For example, the results may, in the judgment of the faculty, indicate a weakness in 
the unit in a very specific area where changing X will correct the problem. Assuming 
change X is implemented, the unit would report, "Change X undertaken." The outcome 
would be assessed again the following year using the same methodology to discover 
whether the change improved achievement. 

Another example, however, would be that the results indicate a weakness, but it is 
not obvious where or what change is needed. In this case, the improvement plan might 
be to establish a committee to examine the situation and report at a later date. The unit 
would report something along the lines of. "committee appointed to review W." Once 
the committee made its report (before the next assessment cycle) a more detailed im- 
provement plan would be drafted, and the outcome would be assessed again. 

If after a second and third assessment cycle the unit is still not achieving the out- 
comes it expects, it is time to examine the outcome (to see if it is realistic), the criteria (to 
see if it is reasonable), and the methodology (to see if it is reliable). 

Action plans for implementing improvements, including timelines and task assignments, 
should be documented into the Unit-Level Effectiveness Database at http://vwvw2.austin.cc.tx.us / 
oiepub/unitlevel.htm. The User name for entering documentation into the database is ACC Units 
and the password is unitgoals. The deadline for documenting improvement plans is June 15 of 
each year. 



7. Impact 




This is the "dessert" of the assessment process. The unit looks back to see how far it has 
come in its quest for quality. The impact statement makes the value of the assessment proc- 
ess visible — not only to the unit, but to everyone. 

s- How has the unit changed as a result of the improvement actions taken by the unit? 

If the unit changed curriculum, the Impact statement is simply a statement of how that 
has effected student or faculty performance. Sometimes, however, there are unexpected 
and presumably unrelated changes as a result of improvement actions. For example, if an 
improvement action included surveying employers for input into skills they wished graduates 
of the program were more proficient in, an unexpected impact of implementing that im- 
provement might be closer relationships with those employers in other ways as well. 

Documentation of the impact of the improvement actions should take place a year follow- 
ing the identification of the outcome. Documentation is entered into the Unit-Level Effective- 
ness Database at http://www2.austin.cc.tx.us./oiepub/unitlevel.htm. The User name for entering 
documentation into the database is ACC Units (two words) and the password is unitgoals 
(one word.) The deadline for entering documentation into the ULE Database is June 15 of 
each year. 
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Outcomes Assessment 
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The Relationship of Types of Planning at an Institution 

From The Departmental Guide and Record Book for Student Outcomes Assessment and Institutional Effectiveness 
by Karen W. Nichols and James 0. Nichols 



In constructing statements of student learning outcomes, it is often difficult to make a dif- 
ferentiation between process/input statements and outcomes/results statements. Karen 
Nichols in her book. The Departmental Guide and Record Book for Student Outcomes Assess- 
ment and Institutional Effectiveness, makes the distinction clear. 
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Nine Principles of Good Practice for Assessing Student Learning 

From The Association for the Advancement of Higher Education (AAHE) Assessment Forum 



1. The assessment of student learning begins with educational values. Assessment is 

not an end in itself but a vehicle for educational improvement. Its effective practice, then, 
begins with and enacts a vision of the kinds of learning we most value for students and 
strive to help them achieve. Educational values should drive not only what we choose to 
assess but also how we do so. Where questions about educational mission and values 
are skipped over, assessment threatens to be an exercise in measuring what's easy, 
rather than a process of improving what we really care about. 

2. Assessment is most effective when it reflects an understanding of learning as multidi- 
mensional, integrated, and revealed in performance over time. Learning is a complex 
process. It entails not only what students know but what they can do with what they 
know; it involves not only knowledge and abilities but values, attitudes, and habits of 
mind that affect both academic success and performance beyond the classroom. As- 
sessment should reflect these understandings by employing a diverse array of methods, 
including those that call for actual performance, using them over time so as to reveal 
change, growth, and increasing degrees of integration. Such an approach aims for a 
more complete and accurate picture of learning, and therefore firmer bases for improv- 
ing our students' educational experience. 

3. Assessment works best when the programs it seeks to improve have clear, explicitly 
stated purposes. Assessment is a goal-oriented process. It entails comparing educa- 
tional performance with educational purposes and expectations - those derived from 
the institution's mission, from faculty intentions in program and course design, and from 
knowledge of students' own goals. Where program purposes lack specificity or agree- 
ment, assessment as a process pushes a campus toward clarity about where to aim and 
what standards to apply; assessment also prompts attention to where and how program 
goals will be taught and learned. Clear, shared, implementable goals are the corner- 
stone for assessment that is focused and useful. 

4. Assessment requires attention to outcomes but also and equally to the experiences 
that lead to those outcomes. Information about outcomes is of high importance; where 
students "end up" matters greatly. But to improve outcomes, we need to know about stu- 
dent experience along the way - about the curricula, teaching, and kind of student effort 
that lead to particular outcomes. Assessment can help us understand which students 
learn best under what conditions; with such knowledge comes the capacity to improve 
the whole of their learning. 

5. Assessment works best when it is ongoing not episodic. Assessment is a process 
whose power is cumulative. Though isolated, "one-shot" assessment can be better than 
none, improvement is best fostered when assessment entails a linked series of activities 
undertaken over time. This may mean tracking the process of individual students, or of 
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cohorts of students; it may mean collecting the same examples of student performance or 
using the same instrument semester after semester. The point is to monitor progress toward 
intended goals in a spirit of continous improvement. Along the way, the assessment process 
itself should be evaluated and refined in light of emerging insights. 

6. Assessment fosters wider improvement when representatives from across the educa- 
tional community are involved. Student learning is a campus-wide responsibility, and as- 
sessment is a way of enacting that responsibility. Thus, while assessment efforts may start 
small, the aim over time is to involve people from across the educational community. Faculty 
play an especially important role, but assessment's questions can't be fully addressed with- 
out participation by student-affairs educators, librarians, administrators, and students. As- 
sessment may also involve individuals from beyond the campus (alumni/ae, trustees, em- 
ployers) whose experience can enrich the sense of appropriate aims and standards for 
learning. Thus understood, assessment is not a task for small groups of experts but a col- 
laborative activity; its aim is wider, better-informed attention to student learning by all parties 
with a stake in its improvement. 

7. Assessment makes a difference when it begins with issues of use and illuminates ques- 
tions that people really care about. Assessment recognizes the value of information in the 
process of improvement. But to be useful, information must be connected to issues or ques- 
tions that people really care about. This implies assessment approaches that produce evi- 
dence that relevant parties will find credible, suggestive, and applicable to decisions that 
need to be made. It means thinking in advance about how the information will be used, and 
by whom. The point of assessment is not to gather data and return "results"; it is a process 
that starts with the questions of decision-makers, that involves them in the gathering and in- 
terpreting of data, and that informs and helps guide continuous improvement. 

8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions 
that promote change. Assessment alone changes little. Its greatest contribution comes on 
campuses where the quality of teaching and learning is visibly valued and worked at. On 
such campuses, the push to improve educational performance is a visible and primary goal 
of leadership; improving the quality of undergraduate education is central to the institution's 
planning, budgeting, and personnel decisions. On such campuses, information about learn- 
ing outcomes is seen as an integral part of decision making, and avidly sought. 

9. Through assessment, educators meet responsibilities to students and to the public. There is 
a compelling public stake in education. As educators, we have a responsibility to the publics 
that support or depend on us to provide information about the ways in which our students 
meet goals and expectations. But that responsibility goes beyond the reporting of such infor- 
mation; our deeper obligation — to ourselves, our students, and society - is to improve. 
Those to whom educators are accountable have a corresponding obligation to support such 
attempts at improvement. 
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Types of Learning Outcomes to Consider 

adapted from Ball State University Assessment Workbook 1992 
Printed by permission of the author 

L. F. Gronlund, in Stating Objectives for Classroom Instruction (1991), provided the following list 
of types of outcomes. The list delineates many of the major areas in which instructional ob- 
jectives might be produced. The specific categories were intended to be suggestive, not ex- 
clusive. 



Knowledge 

• Terminology 

• Specific facts 

• Concepts and principles 

• Methods and procedures 

Application 

• Factual information 

• Concepts and principles 

• Methods and procedures 

• Problem-solving skills 

Thinking skills 

• Critical thinking 

• Scientific thinking 



Understanding 

• Concepts and principles 

• Methods and procedures 

• Written material, graphs, maps, 
and numerical data 

• Problem situations 

General skills 

• Laboratory skills 

• Performance skills 

• Communication skills 

• Social skills 

• Computational skills 

Attitudes 

• Social attitudes 

• Scientific attitudes 



Appreciation 

• Literature, art, music 



Bloom's Classification of Cognitive Skills 

from Ball State University Assessment Workbook 1992 
Printed with permission of the author. 



Bloom's classification of cognitive skills is widely used in instructional planning. The six 
categories are arranged by level of complexity. Use of this or other classification systems is 
recommended to safeguard against a tendency to focus on content coverage and to ignore 
what students should learn to do with content. 



Category 


Definition 


Related Behaviors 


Knowledge 


Recalling or remembering something with- 
out necessarily understanding, using, or 
chanqinq it. 


Define, describe, identify, label, list, 
match, memorize, point to, recall, se- 
lect, state 


Comprehension 


Understanding something that has been 
communicated without necessarily relating 
it to anything else. 


Alter, account for, annotate, calculate, 
change, convert, group, explain, gener- 
alize, give examples, infer, interpret, 
paraphrase, predict, review, summa- 
rize, translate 


Application 


Using a general concept to solve problems 
in a particular situation; using learned ma- 
terial in new and concrete situations. 


Apply, adopt, collect, construct, demon- 
strate, discover, illustrate, interview, 
make use of, manipulate, relate, show, 
solve, use 


Analysis 


Breaking something down into its parts; 
may focus on identification of parts or 
analysis of relationships between parts, or 
recoqnition of organizational principles. 


Analyze, compare, contrast, diagram, 
differentiate, dissect, distinguish, iden- 
tify, illustrate, infer, outline, point out, 
select, separate, sort, subdivide 


Synthesis 


Creating something new by putting parts of 
different ideas together to make a whole. 


Blend, build, change, combine, compile, 
compose, conceive, create, design, for- 
mulate, generate, hypothesize, plan, 
predict, produce, reorder, revise, tell, 
write 


Evaluation 


Judging the value of material or methods 
as they might be applied in a particular 
situation; judging with the use of definite 
criteria. 


Accept, appraise, assess, arbitrate, 
award, choose, conclude, criticize, de- 
fend, evaluate, grade, judge, prioritize, 
recommend, referee, reject, select, 
support 
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Austin Community College Mission Statement 



Austin Community College is an educational institution committed to challenging the hu- 
man mind to explore new ideas and seek new opportunities. The College mission is to pro- 
vide a wide range of high quality educational services that meet the needs of our willing part- 
ners in learning, both those who seek our services and those whom we must seek out. 

Austin Community College operates on the belief that open access to quality postsecond- 
ary educational experiences is vital in a rapidly changing democratic society. Therefore, the 
College exists to provide such educational opportunities to all the people of the Austin Com- 
munity College service area. Hence, Austin Community College maintains an "open door" 
admissions policy, offers a comprehensive variety of postsecondary educational programs, 
and actively seeks to eliminate barriers in the educational process. 

A. Types of Programs 

Austin Community College offers the following types of programs, sen/ices, and instruc- 
tion to fulfill its mission and to satisfy state law for public junior and community colleges: 

[a] Vocational and technical programs of varying lengths leading to certificates or de- 
grees. 

[b] Freshman- and sophomore-level academic courses leading to an associate de- 
gree or serving as the base of a baccalaureate degree program at a four-year in- 
stitution. 

[c] Continuing adult education for academic, occupational, professional, and cultural 
enhancement. 

[d] Special instructional programs and tutorial service to assist underprepared stu- 
dents and others who wish special assistance to achieve their educational goals. 

[e] A continuing program of counseling and advising designed to assist students in 
achieving their individual educational and occupational goals. 

If) A program of technology, library, media, and testing services to support instruction. 

[g] Contracted instructional programs and services for area employers that promote 
economic development. 

B. Intended Results 

[1] The basic result to be produced by the College, in conjunction with other commu- 
nity sectors, is that all service-area adults legally qualified for College services have 
the post-secondary and higher education they need and can use for productive, suc- 
cessful lives. How close the local community is to this goal is a central accountability 
indicator for the College. However, declaration of this goal is not a guarantee of par- 
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ticular services, program admissions, or resource allocations; these are decided 
through the program-review, admissions, and budget processes. 



[2] Accredited preparation shall be provided for as many career areas and university- 
transfer options within the mission of the College as is feasible. Emphasis shall be 
placed on providing post-secondary education (including needed preparation) to 
people who are educationally disadvantaged or are not well-served by other col- 
leges, and on preparation for family-wage careers (either directly or after further 
higher education). 

[3] In addition to mastery of the specific subject-area knowledge and skills needed to 
meet their education-related goals, students completing College programs shall have 
the general skills needed for success in employment and higher education: these in- 
clude dependability, effective communication, gathering and critically assessing infor- 
mation, problem-solving, teamwork, leadership, and a focus on producing results of 
high quality. 

[4] The College shall organize its activities so as to produce as high a level of overall 
value for the community as possible, and shall avoid procedures that waste the 
money or time of students or staff. 

[5] The College shall create a good place to work, to learn, and to otherwise experi- 
ence the higher-education process. 

The Austin Community College Board of Trustees adopted this policy on February 13, 1973, 
and amended it on March 4, 1974, September 2, 1975, January 18, 1982, September 24, 1984, 
February 3, 1997, and May 1, 2000. 
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College-Wide Effectiveness Assessment Plans 



At the institutional level, the conversation about student learning takes place within the 
context of college-wide functions. Effectiveness measures for these functions provide 
evidence of the degree to which the outcomes promised in the institution's mission 
statement are being achieved. According to its mission statement, ACC provides 
vocational and technical programs leading to degrees, certificates, and jobs; freshman and 
sophomore courses in the arts and sciences; continuing adult education; compensatory 
education programs (developmental education courses and adult basic education); and 
assistance to students to help them achieve their educational goals. The Effectiveness 
Measures for these functions are presented here. Most were defined by the President's 
Effectiveness Council, composed primarily of faculty. However, some still in "draft" status 
are currently under discussion in several College Councils. 

Each year, for each college-wide function, data defined in the methodology are 
collected, analyzed, and the results presented in an Office of Institutional Effectiveness 
publication. Effectiveness Update, as well as posted to the OIE web site (http://www2. 
austin.cc.tx.us/oiepub). Additionally, as the data are reviewed, the measures themselves 
may be revised to provide more useful information for these college-wide conversations 
about student learning. 
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Workforce Education Effectiveness Measures 



Purpose: To prepare students for employment in industry and business. 
Approved by President's Effectiveness Council, January 1 998 



Intended Outcome 


Assessment Criteria 


Methodology 


1 . Students in workforce education programs will meet their educational goals. 


1A. Associate degree 
seekers 


Analysis of student data will 
indicate that [of] workforce 
students who indicate their 
educational objective at en- 
try is to "Complete an associ- 
ate's degree," 10% will com- 
plete a degree within 6 
years. Of those who do not 
complete a degree, 50% will 
achieve a certificate or mar- 
ketable skills achievement 
award. 


Longitudinal analysis of 
student intent data and 
program/course comple- 
tion data. 


1 B. Certificate seekers 


Analysis of student data will 
indicate that [of] workforce 
students who indicate their 
educational objective at en- 
try is to "Complete a certifi- 
cate," 10% will do so within 
6 years. Of those who do 
not complete a certificate, 
50% will achieve a degree or 
marketable skills achieve- 
ment award. 


Longitudinal analysis of 
student intent data and 
program/course comple- 
tion data. 


1C. Non-graduates 


Analysis of student data will 
indicate that 90% of non- 
degree seeking students 
who are identified as Mar- 
ketable Skills Achievers and 
complete at least six semes- 
ter credit hours in a work- 
force program will have a cu- 
mulative GPA of at least 
2.00 when they leave ACC. 


Non-degree seeking stu- 
dents are those students 
who indicate on the initial 
application that (1) their 
educational objective at 
time of entry is "Take se- 
lected courses" (question 
4) and (2) their primary 
reason for attending ACC 
is "To improve skill for my 
present job" or To prepare 
for a future job immedi- 
ately after attending 
ACC" (question 2). 
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2. Graduates from workforce programs will find employment in jobs related to their 
fields of study. 


2A. Associate Degree re- 
cipients 


Within one year of gradua- 
tion, 85% of AAS degree re- 
cipients will be employed in 
a job that is directly or 
closely related to their field 
of study, or will be continu- 
ing their education. 


Annual surveys of gradu- 
ates within one year of 
graduation and THECB 
data (if available). 


2B. Certificate recipients 


Within one year of gradua- 
tion, 85% of certificate re- 
cipients will be employed in 
a job that indirectly or 
closely related to their field 
of study or will be continuing 
their education. 


Annual surveys of gradu- 
ates within one year of 
graduation, and THECB 
data (if available). 



Academic Programs Transfer Effectiveness Measures 



Purpose: To prepare students for successful college or university Transfer 

Approved by President's Effectiveness Council, October 1 998 



Intended Outcome 


Assessment Criteria 


Methodology 


1. Transfer Rate 


1A. First-time in College 

(FTIC) students who have 
earned at least 1 5 credit 
hours in baccalaureate trans- 
fer courses will transfer to a 
four-year college or university 
within four years of their ini- 
tial enrollment. 


ACC's transfer rate for FTIC stu- 
dents with a declared major in a 
transfer program will be higher 
than the state average transfer 
rate. 


Comparison of ACC and state- 
wide FTIC transfer rates as 
published in the THECB Com- 
munity Coiiege Transfer Rate 
Study. 


IB. Minority FTIC students 
who have earned at least 15 
credit hours in baccalaureate 
transfer courses will transfer 
to a four-year college or uni- 
versity within four years of 
their initial enrollment. 


ACC's transfer rate for minority FTIC 
students with a declared major in a 
transfer program will be higher 
than the state average transfer 
rate. 


Comparison of ACC and state- 
wide FTIC transfer rates as 
published in the THECB Com- 
munity Coiiege Transfer Rate 
Study. 


2. Transfer Intent Fulfillment 


2A. First-time students who 
indicate an intent to transfer 
and who have earned at least 
15 credit hours in baccalaure- 
ate transfer courses will 
transfer to a four-year college 
or university within six years 
of their initial enrollment. 


65% of first-time students who indi- 
cate an intent to transfer and who 
have earned at least 15 credit 
hours in baccalaureate transfer 
courses will transfer to a four-year 
college or university within six 
years. 


Longitudinal analysis of stu- 
dent intent data collected from 
student applications. 


2B. Minority first-time stu- 
dents who indicate an intent 
to transfer and who have 
earned at least 1 5 credit 
hours in baccalaureate trans- 
fer courses will transfer to a 
four-year college or university 
within six years of their initial 
enrollment. 


65% of minority first-time students 
who indicate an intent to transfer 
and who have earned at least 15 
credit hours in baccalaureate trans- 
fer courses will transfer to a four- 
year college or university within six 
years. 


Longitudinal analysis of stu- 
dent intent data collected from 
student applications. 


3. Success at the Transfer Institution 


3A. ACC Students who trans- 
fer will be in good academic 
standing at the transfer insti- 
tution. 


70% or more of ACC transfer stu- 
dents will be in good academic 
standing at the transfer institution. 


Analysis of transfer data from 
transfer institutions including 
UT at Austin, SWTSU, and 
TAMU. 


3B. ACC students who trans- 
fer will earn GPAs comparable 
to other transfer students at 
the transfer institution. 


The average GPA of ACC transfer 
students and other transfer stu- 
dents at principal transfer institu- 
tions will not be significantly differ- 
ent. 


Analysis of transfer data from 
transfer institutions including 
UT at Austin, SWTSU, and 
TAMU. 
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Adult Education Effectiveness Measures 



Purpose: To improve educational opportunities for adults who lack the level of literacy 
skills requisite to effective citizenship and productive employment. 

Approved by President’s Effectiveness Council, May 99 



Intended Outcome 


Assessment Criteria 


Methodology 


1 . Completion Rate: Adults who need basic education skills necessary for literacy functioning 
will complete the literacy program. 


1.1. Beginning Literacy- 
ABE students will complete 
beginning level adult basic 
education courses. 


15% of Beginning Literacy- 
ABE students will advance 
to intermediate level ABE 
courses. 


Using data reported to the state, 
calculate the percent of all Begin- 
ning Literacy-ABE students who 
advance to the intermediate level 
of adult basic education courses. 


1.2. Beginning Literacy- 
ESL students will complete 
beginning level adult basic 
education courses. 


17% of Beginning Literacy- 
ABE students will advance 
to intermediate level ABE 
courses. 


Using data reported to the state, 
calculate the percent of all Begin- 
ning Literacy-ESL students who 
advance to the intermediate level 
of adult basic education courses. 


1.3 Beginning ABE stu- 
dents will complete begin- 
ning level adult basic edu- 
cation courses. 


20% of Beginning ABE stu- 
dents will advance to inter- 
mediate level ABE courses. 


Using data reported to the state, 
calculate the percent of all Begin- 
ning ABE students who advance 
to the intermediate level of adult 
basic education courses. 


1.4 Beginning ESL stu- 
dents will complete begin- 
ning level adult basic edu- 
cation courses. 


22% of Beginning ESL stu- 
dents will advance to inter- 
mediate level ABE courses. 


Using data reported to the state, 
calculate the percent of all Begin- 
ning ESL students who advance 
to the intermediate level of adult 
basic education courses. 


2. Completion Rate: Adults who complete intermediate bas 
sic education to enable them to benefit from job training a 
productive employment. 


c education will have sufficient ba- 
nd retraining programs and obtain 


2.1 Intermediate ABE stu- 
dents will complete inter- 
mediate level adult basic 
education courses. 


23% of Intermediate ABE 
students will advance to ad- 
vanced level ABE courses. 


Using data reported to the state, 
calculate the percent of all Inter- 
mediate ABE students who ad- 
vance to the advanced level of 
adult basic education courses. 


2.2 Intermediate ESL stu- 
dents will complete inter- 
mediate level adult basic 
education courses. 


23% of Intermediate ESL 
students will advance to ad- 
vanced level ABE courses. 


Using data reported to the state, 
calculate the percent of all Inter- 
mediate ESL students who ad- 
vance to the advanced level of 
adult basic education courses. 
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3. Completion Rate: Adults who desire to continue their education will advance to at least the 
level of completion of secondary school. 


3.1. Advanced ABE students 
will obtain a GED. 


30% of Advanced ABE stu- 
dents will obtain a GED. 


Using data reported to the 
state, calculate the percent of 
all Advanced ABE students 
who obtain a GED. 


3.2. Advanced ESL students 
will meet completion stan- 
dards set by ACC's college- 
wide ESL Task Force. 


24% of Advanced ESL stu- 
dents will meet completion 
standards set by ACC's col- 
lege-wide ESL Task Force. 


Using data reported to the 
state, calculate the percent of 
all Advanced ESL students 
who meet completion stan- 
dards set by ACC's college- 
wide ESL Task Force. 


4. Access 


4.1. Student enrollments in 
ABE classes will meet target 
numbers. 


Enrollments in ABE classes 
will be within 5% of the tar- 
get set in the grant for each 
level of instruction. 


Using data reported to the 
state, compare actual enroll- 
ments in each level of instruc- 
tion with target enrollments 
defined in the grant. 


4.2. Students enrolled in ABE 
classes will meet eligibility cri- 
teria. 


100% of students enrolled 
in ABE classes will meet 
state eligibility criteria. 


Using data reported to the 
state, calculate percentage of 
students enrolled in ABE 
classes who meet state crite- 
ria. 


4.3. Students who take ABE 
classes will complete the ini- 
tial 12 hours of class. 


80% of all students enrolled 
in ABE classes at ACC will 
complete the initial 12 
hours of class. 


Using data reported to the 
state.calculate the percent of 
all students enrolled in ABE 
classes who complete the ini- 
tial 12 hours of class. 



Developmental Education Effectiveness Measures 



Purpose: To provide campus-wide programs and services that assist academically under- 
prepared students so that they will meet their educational goals. 



Intended Outcome 


Assessment Criteria 


Methodology 


1 . Fall-to-Spring Retention Rates 


1.A First-Time in College 
(FTIC) degree seeking stu- 
dents enrolled at ACC in 
the fall semester and who 
are required to take devel- 
opmental courses will re- 
turn the following spring 
semester (Fall-to-Spring 
Retention). 


1A.1 The percentage of FTIC 
degree-seeking students en- 
rolled in developmental educa- 
tion courses in the fall semester 
who return the following spring 
semester will not be signifi- 
cantly different from the per- 
centage of students who were 
not required to take develop- 
mental education courses who 
returned in the Spring. 


Using THECB Annual Data 
Profile report, compare the 
re-enrollment rates of stu- 
dents taking developmental 
courses with the rate of 
those who are not. 




1A.2 Based on data provided 
by the THECB, the percentage of 
FTIC degree-seeking students 
enrolled in developmental edu- 
cation courses in the fall semes- 
ter who return the following 
spring semester will not be sig- 
nificantly different from the 
state average for students en- 
rolled in developmental educa- 
tion courses. 


Using THECB Annual Data 
Profile report, compare ACC 
and statewide average Fall- 
to-Spring retention rates. 


IB. There will be no sig- 
nificant differences by 
demographic group be- 
tween Fall-to-Spring reten- 
tion rates for FTIC degree 
or certificate seeking stu- 
dents enrolled in develop- 
mental education courses 
and those NOT enrolled in 
developmental education 
courses. 


Based on data from the THECB 
Annual Data Profile, for each 
demographic group, the Fall-to- 
Spring return rate for FTIC stu- 
dents enrolled in developmental 
education courses will be within 
±5% of the Fall-to-Spring return 
rate for students NOT enrolled 
in developmental education 
courses 


Using data from the THECB 
Annual Data Profile report, 
compare developmental 
and non-developmental stu- 
dents' average Fall-to-Spring 
retention rates by demo- 
graphic group. 


2. Program Completion Rates 


2A. Students who are re- 
quired to be enrolled in 
Developmental Education 
Courses at ACC will com- 
plete state-mandated re- 
quirements. 


Based on data provided by the 
THECB annually, the percentage 
of ACC students enrolled in de- 
velopmental education courses 
who complete state-mandated 
requirements will be at or above 
the completion rates for those 
students statewide. 


Using THECB data for LBB 
measures, compare ACC 
and statewide percentages. 
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2B. Students who are re- 
quired to be placed in devel- 
opmental courses will com- 
plete their developmental 
requirements. 


Baseline data will be collected to de- 
termine the rates of completion for 
the following groups of developmen- 
tal students: those required to take 
one Developmental Course; those 
required to take two Developmental 
courses; those required to take three 
Developmental courses. 


Using internally generated re- 
ports, determine the comple- 
tion rates by group. 


2C. There will be no signifi- 
cant differences between 
developmental program com- 
pletion rates by demographic 
group. 


The percent of ACC students who 
complete all required developmental 
course work will not be significantly 
different between demographic 
groups. 


Using internally generated re- 
ports, compare developmental 
completion rates by demo- 
graphic group. 


3. Course Completion Rates 


3. Students who are required 
to be placed in developmen- 
tal courses will complete 
their courses at rates similar 
to those taking credit 
courses. 


Course completion rates for develop- 
mental courses will be within ±5% of 
that for credit courses. 


Using internally generated re- 
ports, compare the average 
course completion rate for de- 
velopmental courses with that 
of credit courses. 


4. Developmental to Credit Migration Rates 


4A Students who success- 
fully complete Developmen- 
tal Writing will be successful 
in the next credit-level Eng- 
lish course. 


4A.1 Based on data from the stu- 
dent database, at least 75% of stu- 
dents who complete Writing Skills II 
with a grade of "C" or better and who 
take ENGL 1301 within two years, 
will complete ENGL 1301 with a 
grade of "C" or better. 


4.1a Each semester identify 
cohorts of students who com- 
plete Writing Skills II with a 
grade of X" or better, starting 
in Fall 1997. Each semester 
thereafter, track the cohort en- 
rollment in ENGL 1301. Calcu- 
late the percentage of those 
who complete ENGL 1301 with 
a grade of X" or better within 
two years of their completion of 
Writing Skills II. 


4A.2 Students who complete Writing 
Skills II with a "C" or better and take 
English Comp 1 or English Comp II 
within two years will have course 
grades similar to that of all students 
taking these courses. 


Each year, the average course 
grades of students enrolled in 
English Comp 1 or English 
Comp II who completed Writing 
Skills II with a "C" or better will 
be within ±0.3 of all students 
taking these courses. 


4B. Students who success- 
fully complete Developmen- 
tal Reading will be success- 
ful in the next credit-level 
course. 


4B.1 At least 75% of students who 
complete Reading Skills II with a "C" 
or better will complete a course des- 
ignated to meet TASP guidelines (see 
list below) with a grade of X" or bet- 
ter. 


The grades of at least 75% of 
the students who complete 
Reading Skills II with a "C" or 
better and who take a course 
designated to meet TASP 
guidelines (see below for list) 
within 2 years will be a "C n or 
better. 


4B.2. Students who complete Read- 
ing Skills II with a "C" or better and 
take a course designated to meet 
TASP guidelines (see list below) will 
have course grades similar to that of 
all students taking these courses. 


Each year, the average course 
grades of students enrolled in 
a course designated to meet 
TASP guidelines (see list below) 
who completed Reading Skills 
II with a "C" or better will be 
within ±0.3 of all students tak- 
ing these courses. 
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4C. Students who suc- 
cessfully complete Devel- 
opmental Mathematics 
will be successful in the 
next credit-level math 
course. 


4C.1 At least 75% of stu- 
dents who complete the 
highest level of Develop- 
mental with a "C" or better 
will complete College Alge- 
bra or Trigonometry or Top- 
ics in Mathematics with a 
grade of "C" or better. 


The grades of at least 75% of 
the students who complete the 
highest level of Developmental 
Math with a "C" or better and 
who take College Algebra or 
Trigonometry or Topics in Mathe- 
matics within 2 years will be a 
"C" or better. 




4C.2 Students who com- 
plete the highest level of De- 
velopmental with a "C" or 
better and take College Al- 
gebra or Trigonometry or 
Topics in Mathematics will 
have course grades similar 
to that of all students taking 
these courses. 


Each year, the average course 
grades of students enrolled in 
College Algebra or Trigonometry 
or Topics in Mathematics who 
completed the highest level of 
Developmental Math with a "C” 
or better will be within ±0.3 of all 
students taking these courses. 



Reading 

GOVT 2305 (U.S. Government) 

GOVT 2306 (Texas State and Local Government) 
ENGL 2322 (British Literature I) 

ENGL 2323 (British Literature II) 

ENGL 2327 (American Literature I) 

ENGL 2328 (American Literature II) 

ENGL 2332 (World Literature I) 

ENGL 2333 (World Literature II) 

HIST 1301 (U.S. History I, self-paced) 

HIST 1302 (U.S. History II, self-paced) 
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Student Services Effectiveness Measures 



Purpose: To provide exemplary programs that support the teaching/learning process 
and to increase opportunities for students to define and reach their educational goals. 
Approved by President's Effectiveness Council February, 1999. 



Intended Outcome 


Assessment Criteria 


Methodology 


1 . Enrollment 


The unduplicated head- 
count enrollment of ACC 
will reflect the diversity 
of the college age popu- 
lation the College serves. 


the percent of ACC students 
from each demographic 
group will be within ±5% of 
their proportion in the ser- 
vice area college age popu- 
lation. 


Compare THECB demo- 
graphic data on the service 
area population and the fall 
unduplicated headcount 
enrollment of ACC stu- 
dents. 


2. Applicants 


The applicants who reg- 
ister and attend ACC will 
reflect the demographics 
of the applicant pool. 


At least 75% of each demo- 
graphic group represented 
in the applicant pool will ac- 
tually enroll. 


Compare demographic 
data from the yearly appli- 
cant pool and the annual 
enrollment of first time at 
ACC students. 


3. Financial Aid 


ACC will provide financial 
aid opportunities to stu- 
dents who demonstrate 
the most need. 


At least 50% of the stu- 
dents who receive Pell 
Grants will have an Ex- 
pected Family Contribution 
(EFC) of zero (0). 


Analysis of data from the 
Federal Pell Grant Program 
Student Payment Sum- 
mary. 


4. Loan Default Rate 


The college will maintain 
a loan default rate 5% 
below the maximum 
threshold required by 
law. 


The College's loan default 
rate, as calculated by the 
Default Management Divi- 
sion of the Department of 
Education, will be at least 
5% less than the federally 
defined maximum thresh- 
old. 


Analysis of data from the 
Department of Education's 
Default Management Divi- 
sion's Annual Report to 
ACC. 



Retention Effectiveness Measures 



Purpose: To provide consistent, campus-wide programs and services that increase the 
likelihood that students will remain in college and complete their degree or certificate 
program. 

DRAFT 



Intended Outcome 


Assessment Criteria 


Methodology 


1. Fall-to-Spring Retention Rates 


1A. First-Time in College 
(FTIC) degree seeking stu- 
dents enrolled at ACC in 
the fall semester will return 
the following spring semes- 
ter (Fall-to-Spring Reten- 
tion). 


The percentage of FTIC degree 
seeking students enrolled in the 
fall semester who return the fol- 
lowing spring semester will be 
higher than the state average. 


Using THECB Annual Data 
Profile report, compare ACC 
and statewide average Fall- 
to-Spring retention rates. 


1 B. There will be no signifi- 
cant differences between 
ACC and statewide average 
Fall-to-Spring retention 
rates for FTIC degree or 
certificate seekina stu- 


For each demographic group, 
the percent of FTIC students en- 
rolled in the fall semester who 
return the following spring se- 
mester will be within ±5% of 
their proportion statewide. 


Using data from the THECB 
Annual Data Profile report, 
compare ACC and statewide 
average Fall-to-Spring reten- 
tion rates by demographic 
group. 


2. Fall-to-Fall Retention Rates 


2A. Students enrolled at 
ACC in the fall semester 
will return the following fall 
semester (Fall-to-Fall Re- 
tention). 


The percentage of ACC students 
enrolled in the fall semester who 
return the following fall semes- 
ter will be higher than the state 
average. 


Using THECB Student Migra- 
tion report, compare ACC 
and statewide average Fall- 
to-Fall retention rates. 


2B. There will be no sig- 
nificant differences be- 
tween ACC and statewide 
average Fall-to-Fall reten- 
tion rates by demographic 
group. 


For each demographic group, 
the percent of ACC students en- 
rolled in the fall semester who 
return the following fall semes- 
ter will be within ±5% of their 
proportion statewide. 


Using data from the THECB 
Student Migration report, 
compare ACC and statewide 
average Fall-to-Fall retention 
rates by demographic group. 


3. Completion Rates 


3A. FTIC degree or certifi- 
cate-seeking students will 
complete their degree or 
certificate within three 
years. 


Three-year completion rates of 
FTIC degree-seeking students 
will be at least 5% above the 
state average. 


Using data from the THECB 
Annual Data Profile report, 
compare ACC and statewide 
average completion rates. 


3B. There will be no signifi- 
cant differences between 
ACC and statewide degree 
or certificate completion 
rates of FTIC students by 
demographic group. 


For each demographic group, 
the percent of FTIC students 
who complete a degree or cer- 
tificate will be within ±5% of 
their proportion statewide. 


Using data from the THECB 
Annual Data Profile report, 
compare ACC and statewide 
completion rates by demo- 
graphic group. 
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Continuing Education Programs Effectiveness Measures 



Purpose: The purpose of Continuing Education is to provide courses and training to all individu- 
als who wish to improve their job, career, or personal skills. 

DRAFT 



Intended Outcome 


Assessment Criteria 


Methodology 


1. Students who 
take continuing 
education courses 
will be satisfied 
with those 

courses. 


la. Based on data provided by 
the continuing education 
"Program Evaluation" forms, the 
mean "overall course" rating will 
be no lower than 4 on a scale of 
1 = lowest to 5= highest for 80% 
of all continuing education 
courses evaluated. 


la. At the end of each fiscal year, rat- 
ings for the item, "Overall Rating of 
Course" will be averaged for each 
course (sum of ratings/total number of 
evaluations). The percentage of all 
courses receiving at least a mean 
(average) of 4 on this item will be cal- 
culated. 


1b. Based on data provided by 
the continuing education 
"Program Evaluation" forms, the 
mean rating of the item, 'The 
content assisted me in achieving 
my personal objectives." will be 
no lower than 4 on a scale of 1 = 
lowest to 5= highest for 80% of 
all continuing education courses 
evaluated. 


1b. At the end of each fiscal year, rat- 
ings for the item, "The content assisted 
me in achieving my personal objec- 
tives" will be averaged for each course 
(sum of ratings/total number of evalua- 
tions). The percentage of all courses 
receiving at least a mean (average) of 
4 on this item will be calculated. 


2. Employers who 
contract for ACC 
Customized Train- 
ing will be satis- 
fied with the train- 
ing their employ- 
ees receive. 


2a. Based on data provided by 
the Customized Training 
"Company Evaluation" forms, the 
mean rating of the item, "The 
knowledge/skills acquired in the 
seminar will be valuable to our 
employees." will be no lower 
than 4 on a scale of 1 = strongly 
disagree to 5= stronqly aqree. 


2a. At the end of each fiscal year, rat- 
ings for the item, "The knowledge/ skills 
acquired in the seminar will be valu- 
able to our employees" will be aver- 
aged (sum of ratings/total number of 
evaluations). 


2b. Based on data provided by 
the Customized Training 
"Company Evaluation" forms, the 
mean rating of the item, "The 
seminar/course objectives were 
met" will be no lower than 4 on a 
scale of 1 = strongly disagree to 
5= stronqly aqree. 


2b. At the end of each fiscal year, rat- 
ings for the item, "The seminar/course 
objectives were met " will be averaged 
(sum of ratings/total number of evalua- 
tions). 


3. Students tak- 
ing continuing 
education courses 
to meet licensure 
requirements will 
meet those re- 
quirements. 


Based on data collected by the 
annual continuing education sat- 
isfaction survey, 80% of stu- 
dents reporting they took the 
continuing education course to 
meet licensure requirements will 
also report they met those re- 
quirements. 


Each year the OIE will conduct a survey 
of students who have registered for 
continuing education classes. Results 
on the items related to this criterion 
will be tabulated, and percentages will 
be calculated. 



Assessment Plan Template 



Effectiveness Measures Key Questions 

Purpose: 

• Why does the program exist? 

• What does the program provide students? 

• What can students expect to "get" from this program? 

• What is the major "task" of this program? 



Intended Outcome 


Assessment Criteria 


Methodology 


What results will be achieved? 

What will students be able to 
do? 

What will students know? 
What skills will students 
have acquired? 

What impact will the program 
have on students and the 
community? 


Who will demonstrate results? 

When will results be demonstrated? 

What specific behavior or action will 
demonstrate results? 

Under what conditions will results be 
demonstrated? 

What level of achievement is accept- 
able to identify success? 


Which data will be collected? 

Who is responsible for data 
collection? 

When will the data be col- 
lected? 

What "vehicle" will be used 
for collecting data? 

How will the data be ana- 
lyzed? 

How will the results be re- 
ported? 


Summary and 
Analysis of Data 


Improvement 

Actions 


Impact 


What did you find out? 

Were the criteria met? 

What problems need to be ad- 
dressed? 

What successes were identi- 
fied? 


What will you do next? 

What changes need to be made? 

What other data is needed to make 
improvements? 

What budgetary resources are needed 
to make improvements? 


How has the program 
changed as a result of 
improvement actions? 



available 
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Austin Community College Instructional Units 
Effectiveness Measures for Program 

Purpose: The purpose of our program is 



Intended Outcome 


Assessment Criteria 


Methodology 








Data Summary 
and Analysis 


Improvement Actions 


Impact 









3.1 Planning and Evaluation: Educational Programs 

from Criteria for Accreditation, Section III Institutional Effectiveness 
Southern Association of Colleges and Schools (SACS) 

Reprinted by permission of Dr. James Rogers, COC Executive Director 

Educational activities of an institution include teaching, research and public sen/ice. 
Planning and evaluation for these activities must be systematic, broad based, interrelated 
and appropriate to the institution. The institution must define its expected educational results 
and describe its methods for analyzing the results. The institution must 

1. Establish a clearly defined purpose appropriate to collegiate education 

2. Formulate educational goals consistent with the institution's purpose 

3. Develop and implement procedures to evaluate the extent to which these educa 
tional goals are being achieved 

4. Use the results of these evaluations to improve educational programs, sen/ices 
and operations. 

The institution must develop guidelines and procedures to evaluate educational 
effectiveness, including the quality of student learning and of research and service. This 
evaluation must encompass educational goals at all levels and research and sen/ice 
functions of the institution. The evaluation of academic programs should involve gathering 
and analyzing both quantitative and qualitative data that demonstrate student achievement. 

Measures to evaluate academic programs and general education may include the 
following: 



• Evaluation of instructional delivery 

• Adequacy of facilities and equipment 

• Standardized tests 

• Analysis of theses, portfolios, and recitals 

• Completion rates 

• Results of admissions tests for students applying to graduate or professional 
schools 

• Job placement rates 

• Results of licensing examinations 

• Evaluations by employers 

• Follow-up studies of alumni 

• And performance of student transfers at receiving institutions 



The institution must evaluate its success with respect to student achievement in relation to 
purpose, including, as appropriate, consideration of course completion, state licensing 
examinations, and job placement rates. 
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Outcomes Assessment and SACS 

From Resource Manual on Institutional Effectiveness 
Commission on Colleges Jhe Southern Association of Colleges and Schools 
Reprinted by permission of Dr. James Rogers, COC Executive Director 

“We believe strongly that it is both proper and educationally sound to require that an 
accredited institution state its goals, develop methods by which the achievement of these 
goals can be evaluated, demonstrate that the evaluative information received is utilized in 
the planning process, and finally, use the results of these evaluations to improve educational 
programs, sen/ices and operations. " 

James T. Rogers, Executive Director Commission on Colleges 
Southern Association of Colleges and Schools 

The following are excerpts and page references from the SACS Resource Manual on 
Institutional Effectiveness. 

The Evolving Process of Accreditation 

"...the institutional effectiveness criteria are designed to encourage the establishment 
of an internal planning and evaluation system through which institutional self- 
examination becomes a continuous process rather than a periodic event." p. iii 

"...the institution and each of its units has clearly articulated goals, means for 
evaluating achievement of those goals, and processes for using the results of 
evaluation in work toward institutional improvement." p. iv 

"Ultimately, of course, the greatest benefit of an emphasis on institutional 
effectiveness should be the continuing improvement of quality in educational 
programs and services for students...." p. v 

Assumptions 

"The assessment of institutional effectiveness essentially involves systematic, explicit, 
and documented comparison of institutional performance to institutional purpose." p. 2 

"...the primary shared focus is upon the educational program and services provided 

for students the critical issues for the foreseeable future are certain to include the 

question of whether and to what extent colleges are producing more competent 
students." p. 2 

"Though effectiveness measures will vary across different levels and components of a 
college.. .the processes used for recording and reporting data should be consistent, 
and the entire effort should be systematic in nature." p. 2 

"...both qualitative and quantitative means for evaluation should be employed as 
appropriate. Neither is preferred, but the use of only quantitative or only qualitative 
evaluation procedures would likely diminish the effort." p. 2 
"Worthy of final emphasis is the fundamental assumption that systematic self- 



examination of institutional effectiveness will enhance the quality of decision-making in 
the institution, eventually improving the quality of teaching, scholarship, public service, 
and the overall educational experience of students.” p. 3 

An Analysis of Criteria, Section III: Institutional Effectiveness 

"...procedures used in planning and evaluation should be systematic, comprehensive, 
and clearly related to the purpose and goals of the institution." p. 4 

"Each institution is required to state its expectations regarding the important results of the 
educational process and then to describe the methods that will be used in discovering 
the extent to which those expectations are realized...and the description of methods for 
ascertaining results should include information as to how, where, when, and by whom 
the assessment effort will be accomplished." p.4 

" ...an institution must 

1. ...establish a clearly defined purpose appropriate to collegiate education; 

2. formulate educational goals consistent with the institution's purpose; 

3. develop and implement procedures for evaluating the extent to which these 
educational goals are being achieved; and 

4. use the results of these evaluations to improve educational programs, services 
and operations." 

Components of a Planning and Evaluation Process 



STATEMENT qf pqjRFfrSe 
j^mtagla Flanninjnj> 
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"...the graphic illustration may create the impression of a linear process wherein one neatly- 
packaged set of activities follows another in established sequence. The reality is, of course, that 
both institutions and the individuals who work in them are likely to be involved simultaneously in 
evaluating yesterday's results, grappling with today's problems, and planning for tomorrow's 
possibilities. The processes described are therefore understood to be dynamic and interactive." 
P-5 
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"...the graphic illustration may create the impression of a linear process wherein one 
neatly-packaged set of activities follows another in established sequence. The reality 
is, of course, that both institutions and the individuals who work in them are likely to 
be involved simultaneously in evaluating yesterday's results, grappling with today's 
problems, and planning for tomorrow's possibilities. The processes described are 
therefore understood to be dynamic and interactive." p. 5 

Statement of Purpose 

"While a statement of institutional purpose need not reach an operational level of 
detail, it should include substance sufficient to provide a clear sense of direction for 
the institution and an explicit framework for subsequent statements of goals and 
expected educational results." p. 6 

"...a clear statement of purpose describes in general terms the desired results of 
the ...program...." p. 7 

Formulation of Goals/Definition of Expected Educational Results 

"...the achievement of stated goals must be ascertainable. Consequently, institutions 
should use clear and explicit language when describing their goals. Explicit 
statements use concrete language and action verbs. General adjectives and adverbs 
are noticeably absent, with operational phrases or clauses in their place." p. 8 

Evaluation 

"There has been no more sensitive or more essential aspect of the College 
Commission's institutional effectiveness initiative than the emphasis on 

assessment The institution's task is to identify the specific means of evaluating each 

of the significant goals...." p. 9 

"In describing both expected results and evaluation procedures, institutions should 
strive not only to use explicit language but also 

1. to specify appropriate time frames (e.g., for periodic assessment for 
achievement of particular goals over several years, etc.); 

2. to address both minimum standards and targets for excellence; 

3. to indicate at what level (e.g., program, department, division, institution, etc.) 
the assessment will be conducted and used; and 

4. to designate responsibility for implementing evaluation and for ensuring 
appropriate use of evaluation results. 



Use of Results 

"Following review and discussion of evaluative findings, the institution and/or its 
appropriate units should determine the use of significant findings in improving 
programs and operations." p. 10 

"In the absence of commitment to use evaluation results, all previous steps in the 
planning and evaluation process would become little more than futile exercises 
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which an institution can ill afford, and the institution's planning and evaluation process 
could not be considered adequate." p.10 

"During the process for reaffirmation of accreditation, each institution should be able to 
identify any substantial discrepancies belween statements of purpose and goals and the 
results of assessment and to indicate steps planned or taken to address those 
discrepancies." p. 10 
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